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Abstract 

The Synthetic Vision Systems General 
Aviation (SVS-GA) element of NASA's Aviation 
Safety Program is developing technology to 
eliminate low visibility induced General Aviation 
(GA) accidents through the application of synthetic 
vision techniques. SVS displays present computer 
generated 3 -dimensional imagery of the 
surrounding terrain to greatly enhance pilot's 
situation awareness (SA), reducing or eliminating 
Controlled Flight into Terrain (CFIT), as well as 
Low-Visibility Loss of Control (LVLOC) accidents. 
In addition to substantial safety benefits, SVS 
displays have many potential operational benefits 
that can lead to flight in instrument meteorological 
conditions (IMC) resembling those conducted in 
visual meteorological conditions (VMC). Potential 
benefits could include lower landing minimums, 
more approach options, reduced training time, etc. 
SVS conducted research will develop display 
concepts providing the pilot with an unobstructed 
view of the outside terrain, regardless of weather 
conditions and time of day. 

A critical component of SVS displays is the 
appropriate presentation of terrain to the pilot. The 
relationship between the realism of the terrain 
presentation and resulting enhancements of pilot SA 
and pilot performance has been largely undefined. 
Comprised of coordinated simulation and flight test 
efforts, the terrain portrayal for head-down displays 
(TP-HDD) test series examined the effects of two 
primary elements of terrain portrayal: variations of 
digital elevation model (DEM) resolution and 
terrain texturing. Variations in DEM resolution 
ranged from sparsely spaced (30 arc-sec/2, 953ft) to 
very closely spaced data (1 arc-sec/98 ft). 

Variations in texture involved three primary 
methods: constant color, elevation-based generic, 
and photo-realistic, along with a secondary depth 
cue enhancer in the form of a fishnet grid overlay. 
The TP-HDD test series was designed to provide 
comprehensive data to enable design trades to 
optimize all SVS applications, as well as develop 
requirements and recommendations to facilitate the 
implementation and certification of SVS displays. 


The TP-HDD flight experiment utilized the 
NASA LaRC Cessna 206 Stationaire and evaluated 
eight terrain portrayal concepts in an effort to 
confirm and extend results from the previously 
conducted TP-HDD simulation experiment. A total 
of 15 evaluation pilots, of various qualifications, 
accumulated over 75 hours of dedicated research 
flight time at Newport News (PHF) and Roanoke 
(ROA), VA, airports from August through October, 
2002. This report will present results from the 
portion of testing conducted at Roanoke, VA. 

Introduction 

GA aircraft comprise 85 percent of the total 
number of civil aircraft in the United States of 
America (USA). In a report of the National 
Transportation Safety Board (NTSB) accident 
database [1], GA accounted for 85 percent of all 
accidents and 65 percent of all fatalities. The 
combination of night and IMC increased the 
proportion of fatal to total accidents to 64.3 percent, 
making it the most deadly general aviation flight 
environment. 

The ability of a pilot to ascertain critical 
information through visual perception can be 
limited by various weather phenomena, such as 
rain, fog, and snow. Since the beginning of flight, 
the aviation industry has continuously developed a 
variety of devices, such as attitude indicators, radio 
navigation, and instrument landing systems to 
overcome these issues. Recent advances include 
moving map displays, incorporating advances in 
navigational accuracies from the Global Positioning 
System, and enhanced ground proximity warning 
systems. However, all of the aircraft information 
display concepts developed to date require the pilot 
to perform various additional levels of mental 
model development and maintenance and 
information decoding in a real-time environment 
when outside visibility is restricted [2], 

SVS technology will allow this visibility 
problem to be solved with a visibility solution. By 
providing the pilot with a synthetic vision display 
his SA can be dramatically increased during low 



visibility conditions. These displays employ 
computer-generated terrain imagery to present three 
dimensional, perspective, out the window scenes 
with sufficient information and realism to enable 
operations equivalent to those of a bright, clear day, 
regardless of the outside weather conditions [2, 
through 13]. 

An essential component of all SVS displays is 
the synthetic terrain. SVS terrain provides 
information to the pilot regarding the outside world 
and also serves as the backdrop for integration of 
the other elements of the display (such as flight data 
information, guidance symbology, etc.). Effective 
terrain presentation, that conveys the optimum 
information to the pilot with the lowest mental 
workload, is paramount to successful SVS 
development and implementation. 

Numerous publications [2 through 13] are 
available describing various terrain depiction 
techniques for tactical primary flight displays 
(PFDs) and heads-up-displays (HUDs) and strategic 
navigation display (ND) elements of Multi- 
Function Displays (MFDs). These techniques 
include, but are not limited to, ridge lines, grid 
patterns (equal and non-equal spacing), color-coded 
contour lines, varying color textures based on 
elevation, photorealistic textures, and textures with 
an embedded grid pattern. Textures increase terrain 
realism by increasing the level of detail per 
polygon, thus providing additional cues for position 
and closure rate (height and range) estimates. 

Flight tests have demonstrated that adding a 
textured terrain skin to the EADIs and PFDs gave 
pilots a better awareness of their height above the 
ground. However, references 2 through 1 3 did not 
comprehensively investigate terrain portrayal 
techniques as applied to SVS displays, providing 
only information for certain specific cases with 
limited comparisons. 

Recently conducted work at the University of 
Iowa [11] provides detailed information regarding 
SVS terrain portrayal. In this study, a broad 
spectrum of terrain portrayal techniques were 
examined using several types of experimentation 
methods. These included static and dynamic 
display evaluations combined with piloted 
simulations of a perspective terrain display located 
next to an EADI. The objective of reference 1 1 was 
to establish the minimum effective terrain portrayal 


technique to enable the use of currently certified 
computer platforms with limited capabilities. 
Reference 1 1 provides a wealth of data regarding 
human perception of SVS terrain portrayal 
techniques and shows that terrain resolution and 
texturing significantly affect human subjects’ 
ability to maintain SA. 

The TP-HDD test series extends previous 
research on the effects of DEM resolution and 
texturing and includes real-time piloted simulation 
and flight test evaluations, with integrated terrain 
and symbology, on SVS displays. 

Objectives of Flight Test 

The TP-HDD test series was conducted to 
address several critical aspects of SVS displays, 
which concentrated on core technology issues while 
identifying and addressing key certification issues. 
The objectives of the TP-HDD test series were to: 

1) determine the effect of terrain texturing on 
situation awareness (SA) and pilot performance for 
SVS PFDs; 2) determine the effect of DEM 
resolution on SA and pilot performance for SVS 
PFDs; 3) establish field of view (FOV) 
recommendations for SVS PFDs; 4) demonstrate 
the efficacy of SVS displays for a comprehensive 
spectrum of pilots in both mountainous and flat- 
maritime environments; 5) demonstrate that non- 
instrument rated pilots are able to fly to an 
acceptable level of precision, with minimal training; 
using SVS PFDs with tunnel guidance symbology; 
6) confirm and extend previously-conducted 
simulation results. This paper will partially address 
these objectives. Subsequent publications (NASA 
TPs) will present more complete treatments. 

Method 

To partially address these multiple objectives, 
the flight test involved eight display concepts (one 
without terrain and seven SVS variations), two 
evaluation maneuvers (an en route and a approach 
maneuver), and two pilot group classifications. 

Participants 

A total of 1 5 evaluation pilots (EPs) 
participated in this flight test effort. Seven EPs 
participated in the first part of the flight test (at 
Roanoke, VA.). Eight EPs participated in the 



second part of the flight test (at Newport News, 

VA.). 

For the ROA portion of testing, which is the 
focus of this report, EPs were arranged into two 
groups. Private pilots with less then 400 total hours 
comprised the first group. The second group 
consisted of professional test pilots from NASA and 
Boeing, each having much greater then 1,000 flight- 
hours. All EPs participated in the preceding TP- 
E1DD simulation study [12, 13]. 

Terrain Databases 

Digital Elevation Models 

DEM resolution defines the distance between 
elevation data points (post-spacing) for a given a 
given database. Three specific DEM resolutions 
were investigated during the TP-EiDD experiment 
with the intent to cover a broad range of viable 
DEM options. The low resolution, 30 arc-sec 
(900m/2953ft post-spacing) DEM was selected 
because it is freely available and currently used in 
some industry SVS applications due to the low 
amount of computational power required for 
rendering. The medium resolution, 3 arc-second 
(90m/295ft post-spacing) DEM was selected since 
it is also relatively available, and should be easily 
available in the near future. The highest resolution, 

1 arc-sec (30m/98ft post-spacing) DEM option was 
investigated in this experiment to form an upper 
bound for current consideration. 

It should be noted that higher resolution 
databases are much larger in terms of the overall 
number of data points for a given area of coverage 
with higher computational expenses associated with 
manipulating and rendering this data. It also should 
be noted that the smallest polygon that can be 
created with a given DEM has sides equal to the 
distance between data points. For example, the 
smallest possible polygon employed with the 30 
arc-sec DEM would have sides 2,953 feet long. 
Since the lower resolution DEMs are less 
populated, substantial terrain features might be 
excluded. The possibility of losing entire peaks as 
well as detailed terrain relief in the lower resolution 
databases exists. 

Terrain-Texturing Concepts 

Terrain-texturing refers to the method used to 
color the polygons that comprise the terrain 


database. The three primary texturing concepts 
tested were constant-color (CC), elevation-based 
generic (EBG), and photo-realistic (PR). The 
constant-color texturing concept was developed to 
represent a current industry concept that has 
completed the process of Federal Aviation 
Administration (FAA) certification in the Capstone- 
2 program. This texturing concept requires the 
minimal amount of computational resources for 
rendering. This computational reduction enhances 
the potential use of currently certified avionics 
platforms for SVS applications. 

The elevation-based texturing concept consists 
of twelve equal-height coloring bands. These bands 
correspond to different absolute terrain elevation 
levels, similar to the colors employed for Visual 
Flight Rules (VFR) sectional charts. Lower terrain 
levels are colored with darker colors, higher terrain 
levels are assigned lighter colors. A certain shade 
of green was set to the field elevation. The lightest 
color was set to the highest terrain within 50 nm of 
ROA, approximately 4,000 ft MSL. 

The photo-realistic texturing was derived from 
full color ortho-rectified 4 m satellite imagery data. 
The resulting scene was a highly realistic view due 
to the photographic imagery employed. PR 
texturing requires specialized computer-graphics 
resources that exceed current certified flight 
computer platforms primarily due to the large 
amount of texture memory required to create the 
realistic scene. 

Cultural Feature Data 

For the CC and EBG terrain textured display 
concepts, cultural features, such as roads and rivers, 
were included as objects in the terrain database. 
These objects were unnecessary for the PR terrain 
textured concepts since those features were supplied 
directly through the photo-texture images. 

Fish Net Overlay Concept 

In addition to the primary terrain texturing 
concepts, a fishnet (FN) grid overlay was employed 
for several display concepts. The theory of the FN 
grid involves placing grids of known size within the 
synthetic scene to facilitate pilots’ depth perception. 
The potential benefits of the FN grid used in this 
study are cues for depth perception, distance, 
angular orientation and angular rates. The spacing 
of the FN overlay was 500 ft by 500 ft, regardless 
of the DEM resolution. The FN grid was dual-color 



(gray/white), to compensate for different coloring 
of features within the terrain databases (e.g., lighter 
colors of populated areas for the PR texture). 

Airport Models and Objects 

The ROA airport model included runways with 
all runway markings along with most significant 
airport buildings. Airport buildings were developed 
to appear like the actual buildings they represented 
if viewed from approximately 3 miles. All models 
were placed on top of the underlying terrain 
database. Objects greater then 200 ft high within 20 
nm of ROA were represented by narrow rectangular 
barber-striped pole objects representing their actual 
height and location. 

Generation of Terrain Databases 

Research terrain databases were built and 
rendered using commercial off-the-shelf (COTS) 
tools. Terrain Experts’ TerraVista was used to 
construct the database from digital elevation points 
and add one or more textures. MultiGen- 
Paradigm’s Creator was used to create the three 
dimension (3D) models of airports, buildings, 
obstructions and other man made structures. 

Objects were created in OpenFlight format. The 
finished research terrain databases were rendered 
with Quantum3D/CG2 VTree. A variety of other 
COTS tools were also used including Okino 
PolyTrans, and ERDAS Imagine. 

Resulting Display Concepts 

Eight PFD concepts were employed for this 
flight test effort. The first display concept (shown 
in figure 1) (DC 1) was referred to as the Blue- 
Sky/Brown Ground (BSBG) baseline that replicated 
a basic PFD with no SVS terrain. The remaining 
seven DCs were created through variations of DEM 
and texturing as defined in Table 1. Figure 2 shows 
the CC+FN texture with 30 arc-sec DEM that 
represents an industry application of SVS 
technology (DC 2). Figure 3 shows the EBG 
texture concept with 1 arc-sec DEM (DC 4). Figure 
4 shows the PR texture concept with a 1 arc-sec 
DEM (DC 6). Figures 5 and 6 illustrate the FN 
secondary texture concept applied to the EBG and 
PR texturing methods with the 3 arc-sec DEM (DCs 
7, 8), respectively. While the FN was designed to 
enhance the EBG and PR primary texturing 
concepts, it was required for CC texturing, as no 
terrain information is visible without it. 


Visual inspection of the DCs provided in 
figures 1 through 6 reveals the vast differences 
existing between them. Comparing DC 1 to DC 2, 
the presence of location-specific terrain is evident, 
especially in the distance where terrain rises above 
ownship altitude. DC 2 was considered to be the 
minimal SVS terrain portrayal concept. Comparing 
DC 4 to DC 2, and also to DC 1 , the enhanced 
amount of terrain information being provided 
through the application of terrain coloring is evident 
for the EBG texturing (ignoring for the moment the 
variations in DEM resolution). Valleys and hills 
are easily distinguishable with EBG texturing. 
Photo-realistic texturing, as illustrated by DC 6, 
shows the comparative increase in the amount of 
terrain detail. 

The presence of the FN texturing and effect of 
DEM resolution is illustrated through comparisons 
of DC 4 with DC 7 and DC 6 with DC 8. Generally, 
increasing DEM resolution from 3 to 1 arc-sec 
provides some more noticeable detail for terrain 
within 1 to 3 nm of the aircraft. Note the more 
defined ridgeline in the lower left comer of the 
images for the 1 arc-sec DEM concepts (DC 4 and 
DC 6). Due to the small size of the figures 
presented herein, this observation is somewhat 
compromised. 


Texture\DEM 

30 arc-sec 

3 arc-sec 

1 arc-sec 

CC+FN 

DC 2 



EBG 


DC 3 

DC 4 

PR 


DC 5 

DC 6 

EBG+FN 


DC 7 


PR+FN 


DC 8 



Table 1. Listing of Display Concepts Tested. 


Pilot Selectable FOV and minification factor 

For this experiment, FOV refers to the 
horizontal field of view of the image presented on 
the PFD unless otherwise specified. Vertical FOV 
was adjusted based on horizontal FOV using the 
research PFD’s 4:3 aspect ratio. Pilots could select 
one of four FOVs: 22.5, 30, 60, and 90 degrees. 

The minification factor (MF) is defined as the 
amount of angular compression created when non- 
conformal imagery is displayed and is calculated by 
dividing the FOV by the conformal FOV of the 
display device. For this experiment, MFs tested 
were: 2.0, 2.7, 5.5, and 8.2. 




Symbology 

Basic Symbology 

Basic symbology included on the PFD for all 
DCs tested for both maneuvers included air-data, 
orientation, and guidance information. Air-data 
information was presented by integrated airspeed 
and altitude tapes. A vertical speed indicator was 
included in the integrated altitude tape. A roll 
pointer with magnetic heading digital read-out, and 
a pitch ladder provided heading and attitude 
orientation information. Additional symbology 
components were elements that characterized the 
velocity vector cluster. This cluster employed a 
non-quickened velocity vector that depicted current 
aircraft flight path and track angle with an 
acceleration-along-flight-path indicator (off the left 
finlet of the velocity vector marker). 

Guidance Symbology 

Guidance symbology (see figure 1) was 
presented in the form of course deviation indicators 
and the tunnel in the sky and was displayed to the 
pilots for the approach maneuvers only. Vertical 
and lateral dogbone-shaped path deviation 
indicators provided the pilot with information 
regarding proximity of the aircraft to the center of 
the tunnel. Diamond shaped course deviation 
indicators were provided to show localizer and 
glideslope error. Both the dogbone-shaped path 
deviation indicators and diamond-shaped 
localizer/glideslope error deviation indicators were 
co-located on the same scales. Localizer and 
glideslope error information were generated within 
the research software. 

The tunnel in the sky concept featured 400 feet 
wide by 320 feet tall uniform green boxes depicting 
the desired flight path for the approach scenario, 
providing most of the lateral and vertical path 
guidance. Tunnel spacing was dependent on FOV. 
For the wider FOV’s, the tunnel boxes were closer 
together; for the smaller FOV’s, the tunnel boxes 
were spaced farther apart (i.e., FOV=90°, distance 
between boxes = 965 ft; FOV=30°, distance 
between boxes = 4,685 ft). During turns, if the 
flight path required ha nk angle was greater than 5°, 
the boxes were tilted 20° to cue the turn. 

The pilot-selected FOV was displayed digitally 
to the pilot in the lower left comer of the display 
below the airspeed tape. The size of the digital 
FOV readout was enlarged when FOV was being 


adjusted. After three seconds, the digital FOV 
readout was returned to nominal size. 

Strategic Display 

Strategic terrain display concepts, such as the 
United Parcel Service Aviation Technology 
(UPSAT) MX-20 MFD, augment information 
provided to pilots by SVS perspective displays. 

The presence of the MX-20 in this experiment 
provided the capability to evaluate the integrated 
information supplied by both the PFD and the 
MFD, as well as estimate the relative value of each 
type of display. The MX-20 was used in the terrain 
awareness mode only, and was located in the radio 
stack. On the MX-20 MFD, terrain awareness, 
route information, waypoints, and towers were 
portrayed. All display concepts were evaluated in 
the presence of the MX-20 MFD. See figure 7 for a 
photograph of the MX-20 during approach to 
Roanoke, VA. Terrain more then 2,000 ft below 
the aircraft was black, terrain between 2,000 ft and 
500 ft below was green, terrain between 500 ft 
below and ownship altitude was yellow, terrain at 
or above ownship altitude was red. 

Flight Test Aircraft 

The NASA LaRC Cessna-206 H Stationaire 
(C-206) aircraft was employed for this study (see 
figure 8). Nominally a 6 passenger aircraft, it was 
particularly well suited to support this research 
effort due to its substantial payload carrying 
characteristics combined with a high-capacity 
alternator. 

To support this test, a research pallet was 
fabricated to contain the research computer, power 
inverter, audio-video recorders, and air-data and 
heading reference system (ADAHRS). The 
research pallet was located in place of the middle 
left seat. The research computer was located 
approximately where the rear seat would have been. 
The research pilot’s station was the front right seat. 
See figure 9 for a photograph of the instrument 
panel showing the location of the research PFD and 
MX-20. 

Flight Test Equipment 

Various pieces of research hardware were 
installed in the aircraft to support this flight test. 



The most significant elements are included in this 
report. 

ADAHRS 

A Seagull GIA-2000 ADAHRS was employed 
for this flight test to provide all position, 
orientation, and air data. The GIA-2000 was a 
prototype ADAHRS designed to address the need 
for low-cost equipment to enable SVS-GA 
applications. Utilizing it for this test provided data 
to confirm its efficacy as a low-cost ADAHRS 
suitable for SVS-GA applications. 

The ADAHRS provided aircraft acceleration 
(linear and angular) at 50 Hz. This high-rate data 
was combined with low-rate GPS position (supplied 
at 1 Hz) and pressure-based altitude data through a 
complementary filter to generate high-rate position 
information for the SVS display system. All air- 
data was supplied at 5 Hz. 

Research Computer 

The research computer employed for this flight 
test was a rack- mounted SGI/Integraph ZxlO dual- 
processor graphics workstation PC. The ZxlO had 
dual 1-GHz processors, 2-GB of random access 
memory (RAM), 60 GB hard-drive, and a 3DLabs 
Wildcat 4210 graphics card. The resulting research 
computer provided the capability to evaluate 
advanced SVS terrain portrayal concepts while 
taking advantage of many COTS software products. 

Research PFD 

The research PFD, purchased from Computer 
Dynamics, was a COTS 6” high-bright 4:3 aspect 
ratio Liquid Crystal Display (LCD) operated in 
VGA mode (640x480). The display was 
repackaged for this study. Refer to figure 9 for a 
photograph showing the installation of the research 
PFD. Employing a 24.5” eye reference point 
created a unity horizontal FOV of this device of 
approximately 1 1 degrees. Under these conditions, 
the research PFD provided a resolution of 
approximately 57 pixels per degree. This level of 
resolution approached the level of human 
perception of 60 pixels per degree. 

Flight Test Engineer’s Display 

To provide the ability to control test 
parameters and monitor pilot performance, a 15” 
flight test operator’s display was installed on the 
back of the front-right seat. For this test, the video 
generated for the research PFD was routed through 


a video distribution amplifier and distributed to 
both the research PFD and flight test engineer’s 
display. 

Control Position Transducers 

In order to quantify pilot control activity, 
control position transducers (CPTs) were installed 
on the pitch, roll, yaw, throttle, and pitch-trim 
controls. These transducers were routed to the 
research computer through an Analog to Digital (A 
to D) card and their values were recorded. 

Field of View Control 

Pilots were able to select from four FOVs 
through a rotary knob, located adjacent to the lower 
left comer of the research PFD, or a push-button 
switch, located on the yoke. Selectable FOV 
choices were: 22.5, 30, 60, and 90 degrees. 

Evaluation Maneuvers 

En Route Maneuver 

The en route maneuver required the evaluation 
pilot to maintain assigned heading, airspeed, and 
altitude values at appropriate points. The en route 
maneuver began 19 nm southwest of ROA, with a 
heading of 140° and an indicated airspeed of 100 
knots (KIAS). The en route maneuver was initiated 
at 6,500 ft MSL (approximately 4,000 ft AGL). 
Pilots were required to fly straight and level for 
approximately 2.5 minutes, maintaining heading, 
airspeed, and altitude. With the help of the strategic 
display to identify a fly-by waypoint, the EPs were 
asked to execute a left turn, using 20 degrees of 
bank, to a heading of 050°, while simultaneously 
descending 1,500 ft (over rising terrain). For this 
maneuver, part of the descent took place during the 
90° turn; the rest of the descent was completed 
while maintaining the second target heading. The 
target level-off altitude was 5,000 ft MSL 
(approximately 1,000 ft AGL). Overall, the 
maneuver lasted approximately 5 minutes. 

EPs were asked to vary FOV during the entire 
maneuver to any desired setting. At the end of the 
maneuver, the EPs were asked to cycle through 
FOVs, one more time, to support their evaluations. 

Approach Maneuver 

The approach maneuver employed a straight 
and level right-turn 30-degree localizer intercept 
course for the Instrument Landing System (ILS) 33 
approach into ROA. The touchdown zone elevation 



for runway 33 was 1,179 MSL. The task started 12 
nm south of ROA at 3,000 ft MSL. This initial 
altitude provided approximately 800 ft clearance 
over a ridgeline that was crossed on the initial 
segment to enhance pilot evaluations of SVS 
terrain. With flaps were set to 10 degrees, an 
indicated airspeed of 90 KIAS was to be maintained 
throughout the maneuver. The subject pilots were 
tasked to follow the tunnel guidance symbology, 
roughly following a heading of approximately 300°, 
and join the localizer (roughly 10 nm from the 
threshold) and maintain 3,000 ft MSL until 
intercepting the glide slope at approximately 4.5 
nm, then continue flying the approach to 200 ft 
AGL. EPs were asked to vary FOV during the 
entire maneuver to any desired setting. 

Flight Test Operations 

All flight test research operations were 
conducted in Visual Meteorological Conditions 
(VMC) clear of clouds. The safety pilot (SP) 
occupied the front-left seat and flight test engineer 
occupied the mid-right seat. 

Each EP participated in two consecutive 
research flights lasting approximately 2 hours each. 
The first research flight generated data for the en 
route maneuver block. After an approximately 30- 
minute break on the ground outside of the aircraft, 
the second flight generated data for the approach 
maneuver block. To facilitate flight operations, the 
en route data flights usually took off just after 
sunrise (about 6:30am). The approach data flights 
took off at approximately 9:30am and were 
concluded at approximately 12pm. 

The EP was required to wear a baseball hat 
with the brim adjusted to preclude forward vision 
out the window. This also resulted in highly limited 
outside peripheral vision as well. A baseball hat 
was selected instead of a traditional IFR training 
hood due to the training hood’s restrictive field of 
view requiring the EP to rotate his head to be able 
to scan the MX-20 and research PFDs. 

Prior to each set of research data runs, EPs 
were required to perform at least one training run 
for the en route and approach maneuvers. EPs 
could have additional training runs if needed. 
Usually, only one training run was required. EPs 


were informed as to which DC they were evaluating 
prior to the start of the run. 

During the en route maneuver setup, EPs were 
subjected to some mild spatial disorientation 
maneuvering, lasting approximately 1 minute, 
during which they were required to look at the 
floor. Then, the aircraft was quickly stabilized at 
the desired test condition (altitude, airspeed, and 
heading), EPs were instructed to then look at the 
research instruments (MX-20 and research PFD), 
and control was transferred to the EP. Control was 
transferred back to the SP at the end of the en route 
maneuver after the FOV cycling was complete. 

For the approach maneuver, control was 
transferred to the EPs when guidance boxes (of the 
guidance tunnel) first became visible on the 
research PFD, approximately 1 nm from the start of 
the approach path. Control of the aircraft was 
transferred back to the SP at the end of the run 
(approximately 200 ft AGL). 

EPs filled out run questionnaires between 
evaluation maneuvers. Typically, 5 minutes were 
required to reset for the next evaluation maneuver, 
providing adequate time to complete the 
questionnaires. 

Test Matrix 

Evaluation maneuvers were blocked (en route 
or approach) with DCs being randomized to counter 
learning and fatigue effects. The en route block 
was always conducted first. For this flight test, the 
test matrix included evaluations for statistical 
quantitative data of DCs 1 through 6 and either 7 or 
8. DCs 7 and 8 were the EBGFN-3 and PRFN-3 
concepts. For each EP, only one of those FN 
concepts were tested for statistical data in order to 
reduce the total number of data runs per block (en 
route and approach) to 7 to keep flight durations to 
2 hours. EPs were trained using the DC that was 
not part of their statistical data evaluation set (DC 7 
or 8) to enable qualitative data to be gathered. EPs 
who were trained with DC 7 for the en route 
maneuver, were also trained with DC 7 for the 
approach maneuver. 



Data Analyses and Results 

Due to the large amount of data recorded and 
analyzed in this experiment, it is not practical to 
report on all significant aspects of the flight test in 
this paper. This document focuses mostly on 
texturing effects, while providing more general 
results for all DCs tested. 

Qualitative Data 

Questionnaires were administered after each 
evaluation run and after each EP’s flight pair. Run 
questionnaires included Task Load Index (TLX), 
Situation Awareness Rating Tool (SART), Terrain 
Awareness, Stress, and Cooper-Harper elements. 
Post-flight questionnaires included Situational 
Awareness and Subjective Workload and Relative 
Dominance (SASWORD) comparisons for en route 
and approach maneuvers as well as an overall 
preference. In addition, post- flight questions also 
dealt with preferred FOVs, terrain information as 
provided by the MX-20 and PFD, the value of the 
guidance tunnel (for approach), and other issues. 

Only terrain awareness data are presented for 
this report. For the en route maneuver, pilots 
indicated that they were much more aware of terrain 
for most of the SVS concepts tested than for the 
BSBG DC 1 concept. Terrain awareness ratings for 
the en route maneuver are presented in figure 10. 
This figure illustrates the level of increased terrain 
awareness for all SVS DCs. A statistical analysis 
performed on this data indicates that the effect of 
DC was highly significant. Subsequent post-hoc 
analysis produced two distinct groups. The group 
with the lowest level of terrain awareness consisted 
of BSBG (location-specific terrain awareness 
provided by the MX-20) and CCFN-30. The other 
six DCs were included in the second group. From 
figure 1 0, it can be seen that the level of terrain 
awareness for the CCFN-30 was closer to the 
BSBG then to the other SVS concepts. This agrees 
strongly with other pilots’ comments regarding the 
general relationship between the CCFN-30, BSBG, 
and the other SVS concepts during the relatively 
low- workload en route maneuver. 

Terrain awareness ratings for the approach 
maneuver are presented in figure 11. As was the 
case for the en route data, the effect of DC on 
terrain awareness was highly significant. However, 


for the approach data, subsequent post-hoc analysis 
created three separate groups. The group with the 
lowest terrain awareness consisted again of the 
BSBG and CCFN-30. All the PR textured concepts 
(PR-3, PR-1, and PRFN-3) comprised the 
intermediate group. All of the EBG textured 
concepts (EBG-3, EBG-1, and EBGFN-3), and the 
PR textured concepts without FN (PR-3, PR-1), 
comprised the group with the highest level of 
terrain awareness. The post-hoc groupings related 
well to other pilot comments indicating that the 
EBG textured DCs provided terrain information that 
was easier to interpret, which agrees with 
information contained in reference [11]. Important 
terrain features, such as the location of valleys and 
mountains, may be more difficult for the pilot to 
discern due to the masking effect of trees and other 
elements included in the photo-realistic texturing. 
Specific pilot comments reflected a desire to know 
when they were approaching a ground-based hazard 
without a need to know whether it was rocks, dirt, 
or trees. 

Quantitative Data 

In order to effectively analyze pilot control 
performance, both evaluation maneuvers were 
separated into segments. The seven segments for 
the en route maneuver were: 1) straight and level 
flight, 2) descending turn entry, 3) descending turn, 
4) descending turn exit, 5) constant heading 
descent, 6) transition to straight and level flight, and 

7) straight and level flight. For the approach 
maneuver, the eight segments were: 1) Localizer 
intercept, 2) level turn entry, 3) level turn, 4) level 
turn exit, 5) level localizer tracking, 6) glideslope 
intercept, 7) localizer and glideslope tracking, and 

8) localizer and glideslope tracking below 600 ft 
AGL. 

Pilot performance metrics, that quantified safe 
and desirable levels of pilot control of the aircraft, 
as the percent of the time that the pilots maintained 
desired (Level 1), adequate (Level 2), and below 
adequate performance, were established and 
analyzed. Level 1 performance for the en route 
maneuver required the pilot to maintain airspeed 
within +/- 1 0 kts, altitude within +/- 1 00 ft, and 
heading and ha nk angle within +/-10 degrees, when 
appropriate. For the initial portion of the approach 
maneuver, desired path performance was +/- 100 ft 



laterally and +/-80 ft vertically, with +/- 10 kts of 
airspeed error. When the localizer course was 
captured, desired lateral performance was changed 
to be +/- 1 dot of localizer error. When the 
glideslope was captured, the desired vertical 
performance was changed to +/- 1 dot of glideslope 
error. This was done to enable the application of 
more established range-varying performance data 
for analysis. Adequate performance (Level-2) was 
considered to be twice desired limits. 

En Route 

The percentage of time pilots were able to 
maintain Level- 1 performance, for the entire 
relatively low-workload en route maneuver, is 
presented in figure 12 as a function of DC. The 
only information elements available to the pilot for 
task accomplishment consisted of airspeed, altitude 
and heading cues (no tunnel or other guidance 
cues). Comparison of the SVS DCs with the BSBG 
show that, in general, pilots were able to remain 
within Level- 1 performance approximately 97% of 
the time with a standard deviation of 4.8%. All of 
the SVS concepts, except the PR1, produced pilot 
performance results equivalent to the BSBG 
baseline. 

An analysis of variance (ANOVA) was 
conducted on percentage of time within Level- 1 
performance with DC and pilot type as independent 
variables. Neither DC nor pilot type significantly 
affected the percentage of time within Level- 1 . 
Although not statistically significant, the mean time 
within Level- 1 for the PR1 DC dropped to 
approximately 92%. In addition, the standard 
deviation for the PR1 also increased, becoming 
almost twice the average for all DCs, with 3 of the 7 
pilots achieving Level- 1 performance less than 90% 
of the time. Of these 3 pilots, 2 were from the low- 
time VFR group and one was from the high-time 
group. This result could be indicative of the 
increased amount of terrain detail due to the photo- 
realistic texturing combined with the high- 
resolution DEM, slowing the pilot’s scan of the 
control parameters (airspeed, altitude, heading) for 
the PR1 DC. This result agrees with results from 
reference [11]. 

Approach 

For the higher-workload approach task, the 
percentage of time pilots were within Level- 1 
performance for segment 7 is presented in figure 


13. The information elements available to the pilot 
for task accomplishment consisted of the tunnel, the 
course deviation indicators and airspeed cues. 

Pilots were able to maintain Level- 1 performance 
an average of 85% of the time with a standard 
deviation of 13.34%. An ANOVA performed on 
performance data for the entire approach task with 
DC and pilot type as independent variables revealed 
that DC did not significantly affect time within 
Level-1 (F(7,29)=0.542, a=0.796), but pilot type 
was highly significant (F(l,29)=9.23, a =0.005). 
The effect of pilot type on pilot performance for the 
entire approach task (averaged over all DCs) is 
presented in figure 14, where it can be seen that the 
low-time VFR pilots achieved Level- 1 performance 
8 1 % of the time, while the high-time pilots 
achieved Level- 1 performance 95% of the time. 
Although the effect of tunnel guidance has been 
demonstrated to be highly effective towards 
improving low-time pilot performance [13], pilot 
skill and experience still are highly significant 
factors. 

The FAA Instrument Rating Practical Test 
Standards (PTS) [14] metric employed for 
evaluating performance on the ILS final approach 
segment allows no more than three-quarter-scale 
deflection of either the localizer or glide slope 
indications, while maintaining the specified 
airspeed to within 10 knots. For this test, low-time 
VFR pilots were only able to keep maximum 
glideslope error (averaged over all DCs) to within 
75% of maximum deflection for 50% of the 
approaches conducted as illustrated in figure 15. 
Maximum airspeed error (averaged over all DCs) 
exceeded +/-10 kts for 80% of the approaches 
conducted and exceeded +/-19 kts for 5% of the 
approaches conducted as illustrated in figure 16. 

All of the approaches conducted by low-time VFR 
pilots controlled maximum localizer deviation to 
within 30% of maximum deflection (not shown). 

It should be noted that data analyzed herein 
were digital in nature. Short increases in airspeed, 
resulting from turbulence, could introduce effects 
that might not be observed through cross-cockpit 
viewing of a conventional round dial airspeed 
indicator by an FAA examiner. Still, vertical path 
error and speed management emerged as areas 
requiring improvement for low-time pilots to meet 
IFR practical test standards. 



The result that DC did not significantly affect 
pilot’s performance may be attributed to the higher- 
workload task. Pilot comments regarding the 
approach task indicated that they were focusing 
more on the guidance symbology and not on the 
background terrain, explaining the lack of 
significant effect of terrain on pilot’s performance. 
This comment is interesting since EPs provided 
definite responses regarding terrain awareness. 

During this flight test, some issues were 
encountered regarding the characteristics of some of 
the symbology elements. Recall that the motion of 
the velocity vector was dependent on blending low- 
rate position with high-rate acceleration data. 

While the behavior of the velocity vector was 
deemed acceptable for this study, several options 
such as low-pass filtering with pitch-rate 
quickening, are available to improve its 
characteristics. Additionally, EPs were instructed 
to rely only on the airspeed data and acceleration 
indicator arrow, and not the tachometer, to manage 
airspeed. While this strategy appeared acceptable 
during pre-testing, improvements in the acceleration 
arrow performance, the inclusion of the tachometer, 
or both, could have improved airspeed control. 

Lastly, the size of the guidance tunnel also 
affected pilot’s performance (tunnel guidance was 
presented for all DC tested for the approach 
maneuver). For the given size of the tunnel (i.e. 

400 ft wide by 320 ft tall), the guidance tunnel 
becomes larger then the glideslope beginning at 
approximately 600 ft AGL. By the end of the 
approach (i.e. 200 ft AGL), the tunnel was 
approximately 2.7 times (5.4 dots) as large as full- 
deflection of the glideslope. This visual cue could 
have contributed to the low-time pilots’ apparent 
lack of awareness of large glideslope deviations, as 
presented by the glideslope error diamond. All of 
the approaches conducted by the low-time pilots 
produced maximum vertical deviations that 
remained within the tunnel boundaries (i.e. +/-160 
ft) and yet exceeded glideslope tolerances. 

Field of View 

One objective of this flight test effort was to 
establish recommended FOV use for SVS GA 
applications. One factor affecting FOV use was the 
need to keep the velocity vector on the display. 
Significant crab angles were observed for both en 


route and approach maneuvers primarily due to the 
airspeeds employed (i.e. 100 kts and 90 kts). Cross 
wind conditions encountered were considered mild. 
In addition, turbulence effects, combined with the 
natural flight dynamics of the aircraft, produced 
substantial motion of velocity vector position. 
Occasionally, pilots were able to employ lower 
FOVs (30 degrees), to enhance their view of the 
runway during the latter stages of the final 
approach. However, due to the dynamics of the 
aircraft combined with the operating speeds, 
resulting measured FOVs typically ranged near 60 
degrees. In post-block questionnaires, all pilots 
selected 60 degrees FOV as the most preferred for 
the approach maneuver. The average of most 
preferred FOV for the en route maneuver was 64 
degrees. Therefore, the recommended FOV for 
SVS GA applications is 60 degrees. Removal of 
pilot-selectable FOV control may not impose 
substantial restrictions on the utility of these 
displays due to characteristics inherent to GA 
aircraft. This recommendation is counter to 
reference [2] that involved testing with large 
transport aircraft. However, lower FOVs, such as 
30 degrees, could still be useful for calm operating 
conditions, providing increased utility during latter 
stages of final approach. 

Concluding Remarks 

SVS displays have the ability to provide 
enhanced terrain awareness. A continuing objective 
for SVS displays is the appropriate integration of 
SVS terrain with other elements of the display to 
provide optimum pilot SA and workload. Evidence 
is provided in this study to further substantiate the 
idea that high-resolution photo-realistically textured 
terrain databases and high-resolution digital 
elevation models may combine to detract from 
optimal pilot performance (due to the extreme level 
of detail provided by these types of SVS displays) 
for the maneuvers tested. However more data is 
required to draw final conclusions. 

The use of color and texture, as employed for 
the elevation-based generic and photo-realistic 
texturing concepts, has the potential to greatly 
enhance pilot’s terrain awareness over that provided 
by a constant-color with fishnet grid terrain 
portrayal or a standard blue-sky/brown-ground 
PFD. Low-time pilots were not capable of flying 



ILS approaches to established IFR PTS criteria as 
indicated by their maximum glideslope and airspeed 
errors. However, enhancements in symbology 
(velocity vector dynamics and guidance tunnel size) 
and/or the addition of a tachometer could provide 
needed improvements. High-time pilots exceeded 
the IFR PTS criteria for ILS approaches as tested. 

Field of view recommendations from this test 
indicated that 60 degrees was the most preferred for 
approach and would provide the most utility 
because of the substantial movements of the 
velocity vector typical in GA flight operations. 

Pilot selectable field of views, that include 30 
degrees, could be beneficial in calm operating 
conditions on final approach. 

Overall, a comprehensive evaluation of 
specific components (DEM/texture) of SVS terrain 
portrayal methods has been conducted through an 
extensive simulation and flight-test effort. Results 
indicated pilots were able to use the SVS displays 
effectively, with dramatically increased terrain 
awareness. In general, all SVS concepts tested 
provided similar results with some other study data 
[11] suggesting that ultimate terrain portrayal 
fidelity (photo-realism) might not be as important 
as effective terrain portrayal presentation 
(elevation-based generic). One potential concept 
that is recommended for further evaluations would 
be a hybrid EBG/Photo-realistic texturing method 
blending high-detail cultural and feature data with 
the easy to perceive elevation-based generic 
texturing. Such a hybrid concept could combine 
positive features from both texturing concepts. 

Subsequent reports resulting from the TP-HDD 
test series will focus more fully on the remaining 
study objectives. Future work should address the 
issues of SVS terrain integration with a focus on the 
interactions between terrain fidelity, symbology, 
and useable FOVs. 
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Figures 



Figure 1. Blue-Sky Over Brown-Ground 
Baseline (DC 1) With All Symbology Elements 



Figure 2. Constant Color Texture with Fishnet 
Overlay and 30-arcsec DEM (DC 2) 



Figure 3. Elevation-based Generic Texture with 
1-arcsec DEM (DC 4) 



Figure 4, Photo-realistic Texture with 1-arcsec 
DEM (DC 6) 



Figure 5. Elevation-Based Generic Texture with 
Fishnet overlay and 3 arc-sec DEM (DC 7) 










Figure 6. Photo-realistic Texture with Fishnet 
Overlay and 3 arc-sec DEM (DC 8) 



Figure 7. Photograph of MX-20 during 
approach to Roanoke, VA. 




Figure 9. Experimental Instrument Panel 
During Final Implementation 



Figure 10. En Route terrain awareness for all 
DCs tested (0=no location-specific terrain 
awareness). 



Figure 8. Cessna 206-H Stationaire (NASA 504). 


Figure 11. Approach terrain awareness for all 
DCs tested (0= no location-specific terrain 
awareness). 
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Figure 12. Percent time pilots maintained Level- 
1 performance for the entire En Route 
maneuver. 



Figure 13. Percent time pilots maintained Level- 
1 performance during Segment 7 of final 
approach. 
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Figure 14. Percent time pilots maintained Level- 
1 performance for the entire approach task 
(averaged over all DCs). 
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Figure 15. Histogram of Maximum Glideslope 
Error (averaged over all DCs ) for Low-time 
VFR Pilots for Segment 7 of final approach. 



Figure 16. Histogram of Maximum Airspeed 
Error (averaged over all DCs ) for Low-time 
VFR Pilots for Segment 7 of final approach. 


